Skip to content

sans-io parsers - #139

Draft
lovelydinosaur wants to merge 18 commits into
mainfrom
sansio
Draft

lovelydinosaur wants to merge 18 commits into
mainfrom
sansio

Conversation

@lovelydinosaur

@lovelydinosaur lovelydinosaur commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Parsers

Hip provides a little Sans I/O HTTP client parser, inspired by h11.

Usage

The parser has a fairly simple little API.

There are 3 methods for sending parts of the request, and 3 methods for receiving the response. Each set needs to be called in order, although sending and receiving operations can be interleaved.

The parser class does not perform any I/O itself. Instead the send methods return data that needs to be sent on the wire, while the recieve methods explicitly signal if more data needs to be fed into the parser.


Sending the request.

.send_request_line(method, target, protocol)
    method - A bytes instance representing the request method, such as `GET`.
    headers - A bytes instance representing the request URI, such as `/`.
    protocol - The bytes instance b'HTTP/1.1'.
    return - The bytes representation of the data that needs to be sent on the wire.
  
.send_headers(headers)
    headers - A list of `(byte, byte)` tuples representing the request headers.
    return - The bytes representation of the data that needs to be sent on the wire.
  
.send_body(headers)
    body - A bytes instance of data to be sent in the request body.
           To finalise the request, send b''.
    return - The bytes representation of the data that needs to be sent on the wire.
  

Receiving the response.

.recv_response_line()
    return - A three tuple of `(bytes, int, bytes)` representing the status line of the response.
             These are the protocol string (eg b'HTTP/1.1'), status code (eg 200), and reason phrase (eg b'OK').
             Returns `None` if the parser does not have enough data yet. 
  
.recv_headers()
    return - A list of `(byte, byte)` tuples representing the response headers.
             Returns `None` if the parser does not have enough data yet. 
  
.recv_body()
    body - A bytes instance of data representing part in the request body.
           Returns b'' to finalise the response.
           Returns `None` if the parser does not have enough data yet.
  

Feeding data to the parser.

.feed(data)
    data - A bytes instance of data representing incoming data to be fed to the parser.
           The b'' byte string is used to indicate EOF.
  

Example

The following demonstrates a little example of using standard python sockets to send a single request, and parse the response. This uses the standard library, plus the truststore package for dealing with SSL verification.

import socket
import truststore

host = "httpbin.org"
buffer_size = 65536

# Create a standard socket connection to <host>
# Use the truststore package for SSL verification.
ctx = truststore.SSLContext(ssl.PROTOCOL_TLS_CLIENT)
socket = socket.create_connection(host, 443)
socket = ctx.wrap_socket(socket, server_hostname=host)

try:
    parser = hip.parser.HTTPClient()

    # Send the HTTP request.
    data = parser.send_request_line(b"GET", b"/", b"HTTP/1.1")
    data += parser.send_headers([
        (b'Host', host.encode('ascii')),
        (b'User-Agent', 'hip/developer')
    ])
    data += parser.send_body(b'')
    socket.sendall(data)

    # Receive the HTTP response.
    while (response := parser.recv_response_line()) is None:
        parser.feed(socket.recv(buffer_size))
    while (headers := parser.recv_headers()) is None:
        parser.feed(socket.recv(buffer_size))
    while (part := parser.recv_body()) != b'':
        if part is None:
            parser.feed(socket.recv(buffer_size))

finally:
    # Close the socket connection.
    socket.close()

@lovelydinosaur

Copy link
Copy Markdown
Member Author

@encode/hip - This is starting to be first-pass reviewable...

Here's what I'm planning on adding next...

  • Code coverage. (This is where the tests are currently failing)
  • Add ParserError for exception cases.
  • Add .is_keep_alive property to indicate if the connection is reusable.
  • Fix HEAD responses, ensuring that 0 bytes are used.

I think that's the essential functionality covered at that point?

After that, the hip.parsers.HTTPServer() class needs implementing.


Lower priority there's also the following bits of HTTP/1.1 edginess...

  • Expect: 100-continue / 100 Continue / 103 Early Hints - Allow multiple 1xx informational responses parts.
  • Connection: upgrade / 101 Switching Protocols - Terminate after initial part of response.
  • CONNECT / 2xx - Terminate after initial part of response.
  • 'Transfer-Encoding: chunked' - chunk extensions
  • 'Transfer-Encoding: chunked' - trailing headers

Comment thread src/hip/parsers/sansio.py Resolved
DevNeto07

This comment was marked as resolved.

Comment thread src/hip/parsers/sansio.py Resolved
Comment thread src/hip/parsers/sansio.py Resolved
Comment thread src/hip/parsers/sansio.py
if self._client != SEND_START:
raise RuntimeError("Called 'recv_request_line' out of sequence.")

if (data := self._parser.read_until(b"\r\n", MAX_RESPONSE_LINE_SIZE)) is None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this use MAX_REQUEST_LINE_SIZE instead of MAX_RESPONSE_LINE_SIZE?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants